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Claims 

1 . (Currently Amended) A method of constructing a model operable to generate 
one or more job performance criteria predictions based on input pre-hire information, the 
method comprising: 

electronically collecting pre-hire information from a plurality of applicants wherein at 
least some of the pre-hire information is collected from at least one of the applicants who 
responds directly on an electronic device to provide pre-hire applicant responses to 
questions ; 

collecting post-hire information for the applicants based on job performance of the 
applicants after hire^ ^ wherein collecting post - hir e information compri s e s receiving 
payroll information for the applicants via a nctworlc, determining a termination date for 
an applicant from th e payroll information receiv e d via the network^ and determining 
tenur e of the applicant by comparing th e t e rmination date with a hiring date of the 
applicant; and 

via information-theoretic feature selection^ choosing questions from the pre-hire 
information as features for which respective pre-hire applicant responses serve as inputs 
to the model; and 

from the pre-hire information and the post-hire information, generating training an 
artificial intelligence-based predictive model in a computer-readable medium with observed 
pre-hire applicant responses for the chosen features , wherein the artificial intelligence- 
based predictive model is operable to generate one or more job performance criteria 
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predictions comprising tenure based at least on input pre-hire information from new applicants 
corresponding to the chosen features , whereby the one or more job performance criteria 
predictions are usable as a basis for a hiring recommendation or other employee selection 
information. 

2. (Currently Amended) A computer-readable medium comprising computer- 
executable instructions for performing a method of constructing a model operable to generate 
one or more job performance criteria predictions based on input pre-hire information, the 
method comprising: 

electronically collecting pre-hire information from a plurality of applicants wherein at 
least some of the pre-hire information is collected from at least one of the applicants who 
responds directly on an electronic device to provide pre-hire applicant responses to 
questions : 

collecting post-hire information for the applicants based on job performance of the 
applicants after hirci , wh e r e in coll e cting post hir e information compris e s r e c e iving payroll 
information for th e applicants via a n e twork, d e t e rmining a t e rmination dat e for an applicant 
from th e payroll information r e c e iv e d via the n e twork, and d e t e rmining t e nur e of the applicant 
by comparing th e t e rmination dat e with a hiring dat e of th e applicant; and 

via information-theoretic feature selection, choosing questions from the pre-hire 
information as features for which respective pre-hire applicant responses serve as inputs 
to the model; and 
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from the pre-hire information and the post-hire information, generating training an 
artificial intelUgence-based predictive model witii observed pre-hire applicant responses for 
tlie cliosen features^ wiierein tlie artificial intelligence-based predictive model is operable 
to generate one or more job performance criteria predictions comprising tenure based at least 
on input pre-hire information from new applicants corresponding to the chosen features . 

3. (Canceled) 

4. (Canceled) 

5. (Original) The method of claim 1 further comprising: 

limiting the applicants for the model to those with a particular occupation; and 
constructing the model as an occupationally-specialized model. 

6. (Original) The method of claim 1 wherein the model accepts one or more inputs, 
the method further comprising: 

identifying in the pre-hire information one or more characteristics that are ineffective 
predictors; and 

omitting the ineffective predictors as inputs to the model. 

7. (Original) The method of claim 1 wherein the pre-hire information comprises 



Page 4 of 22 



GLM:kab 7/29/08 858793 Attorney Reference Number 5437-60780-01 

PATENT Application Number 09/921,993 

one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are ineffective 
predictors; and 

providing an indication that the characteristics no longer need to be collected. 

8. (Previously Presented) The method of claim 1 wherein job performance criteria 
predictions comprise a prediction indicating whether a job candidate will be voluntarily 
terminated. 

9. (Previously Presented) The method of claim 1 wherein job performance criteria 
predictions comprise a prediction indicating whether a job candidate will be eligible for rehire 
after termination, 

10. (Previously Presented) The method of claim 1 wherein the pre-hire information 
comprises one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are ineffective 
predictors; 

responsive to identifying the ineffective predictors, collecting new pre-hire information 
not including the ineffective predictors; and 

building a refined model based on the new pre-hire information. 
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1 1 . (Original) The method of claim 10 further comprising: 

adding one or more new characteristics to be collected when collecting the new pre-hire 
information, 

12. (Original) The method of claim 1 1 further comprising: 
evaluating the effectiveness of the new characteristics. 

13. (Canceled) 

14. (Canceled) 

15. (Currently Amended) A method for constructing an artificial intelligence- 
based employment selection process based on pre-hire information comprising personal 
employee characteristics and post-hire information comprising employee job performance 
observation information, the method comprising: 

via information-theoretic feature selection, choosing questions from the pre-hire 
information as chosen features for which respective pre-hire applicant responses serve as 
inputs to predictive artificial models; 

generating a plurality of predictive artificial intelligence models accepting the chosen 
features as inputs, b ased on the pre-hire and post-hire information, wherein at least two of the 
artificial intelligence models are of different feed-forward types and different architectures; 
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testing effectiveness of the models to select an effective model; and 
applying the effective model to predict post-hire information not yet observed, whereby 
the post-hire information not yet observed that is predicted by the effective model can be a 
basis for a hiring recommendation or other employee selection information. 

16. (Canceled) 

17. (Previously Presented) The method of claim 1 5 wherein at least one of the 
models is an expert system. 

18. (Canceled) 

19. (Canceled) 

20. (Canceled) 

21. (Original) The method of claim 15 further comprising: 
identifying at least one of the models as exhibiting impermissible bias; and 
avoiding use of the models exhibiting impermissible bias. 

22. (Canceled) 
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23, (Currently Amended) A computer-implemented method of refining an 
artificial-intelligence based employee performance selection system, the method comprising: 

collecting information via an electronic device presenting a set of questions to 
employment candidates, wherein the questions are stored in a computer-readable medium; 

collecting post-hire information for the employment candidates based on job 
performance of the employment candidates after hire, wherein collecting post-hire information 
comprises receiving payroll information for the employment candidates via a network, 
determining a termination date for an employment candidate from the payroll information 
received via the network, and determining tenure of the employment candidate by comparing 
the termination date with a hiring date of the applicant; 

testing effectiveness of at least one of the questions in predicting tenure , wherein 
testing effectiveness comprises considering various sets of permutations of predictive 
items comprising tlie at least one of the questions, wlierein the at least one of tiie 
questions alone is ineffective but is effective in combination with one or more other items : 
and 

responsive to determining the question is ineffective in predicting tenure, deleting the 
question from the computer-readable medium. 



24. (Canceled) 
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25. (Currently Amended) A computer-readable medium comprising a predictive 
model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for a job 
applican t wherein at least one of the inputs is chosen via information-theoretic feature 
selection and corresponds to a question for which pre-hire applicant responses have been 
collected : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, 

wherein the predictive model is an artificial intelligence-based model constructed from 
pre-hire data electronically collected from a plurality of employees and post-hire data, and the 
model generates its predictive outputs based on the similarity of the inputs to pre-hire data 
collected for the plurality of employees and their respective post-hire data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected from the group consisting of: 

whether the job applicant will be involuntarily terminated, and whether the job applicant 
will be eligible for rehire upon termination. 

26. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating a rank for the job applicant. 
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27. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to other applicants, 

28. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to the plurality of employees. 

29. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating probability of group membership for the job 
applicant. 

30. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating predicted tenure for the job applicant. 

31. (Canceled) 

32. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating predicted number of accidents for the job 
applicant. 
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33. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating whether the applicant will be involuntarily 
terminated. 

34. (Original) The computer-readable medium of claim 25 wherein the predictive 
model comprises a predictive output indicating whether the applicant will be eligible for rehire 
after termination. 



35. (Canceled) 



36. (Canceled) 
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37. (Previously Presented) A computer-readable medium comprising a refined 
predictive artificial intelligence-based model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for a job 
applican t, wherein at least one of the inputs is chosen via information-theoretic feature 
selection and corresponds to a question for which pre-hire applicant responses have been 
collected : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs; 

wherein the one or more predictive outputs are configured to predict whether the job 
applicant will be involuntarily terminated or whether the job applicant will be eligible for 
rehire upon termination, and 

wherein the predictive model is constructed from pre-hire data electronically collected 
from a plurality of employees and post-hire data, wherein the pre-hire data is based on a 
question set refined by having identified and removed one or more questions as ineffective. 



38. (Canceled) 



39. (Canceled) 
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40. (Currently Amended) An apparatus for assisting in determining the suitability 
of an individual for employment by an employer, the apparatus comprising: 

a computer-readable medium comprising a predictive model comprising: 
inputs for accepting one or more characteristics based on pre-hire information for a job 
applican t wherein at least one of the inputs is chosen via information'theoretic feature 
selection and corresponds to a question for which pre-hire applicant responses have been 
collected : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, wherein the predictive model is an artificial intelligence-based 
model constructed from pre-hire data electronically collected from a plurality of employees 
and post-hire data, and the model generates its predictive outputs based on the similarity of the 
inputs to pre-hire data collected for the plurality of employees and their respective post-hire 
data; wherein the one or more predicted job effectiveness criteria comprise at least one selected 
from the group consisting of: whether the job applicant will be involuntarily terminated, and 
whether the job applicant will be eligible for rehire upon termination; 
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electronic data interrogator means for presenting a first set of a plurality of means for 
questioning to the individual; 

electronic answer capturer means for electronically storing answers by the individual to 
at least a selected plurality of the first set of means for questioning presented to the individual; 

electronic predictor means responsive to the stored answers and for predicting at least 
one post-hire outcome if the individual were to be employed by the employer, the predictor 
providing a prediction of the outcome based upon correlations of the stored answers with 
answers to sets of means for questioning by other individuals for which post-hire information 
has been collected; 

electronic results provider means for providing an output indicative of the outcome to 
assist in determining the suitability of the individual for employment by the employer. 

41. (Currently Amended) An apparatus for assisting in determining the suitability 
of an individual for employment by an employer, the apparatus comprising: 

an electronic device for presenting a first set of a plurality of means for questioning to 
the individual; 

means for electronically storing answers by the individual to at least a selected plurality 
of the first set of means for questioning presented to the individual; 
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means responsive to the stored answers and for predicting at least one post-hire outcome 
if the individual were to be employed by the employer, the predictor providing a prediction of 
the outcome based upon correlations of the stored answers with answers to sets of means for 
questioning by other individuals for which post-hire information has been collected; and 

means for providing an output indicative of the outcome to assist in determining the 
suitability of the individual for employment by the employer, 

wherein the apparatus comprises a predictive model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for a job 
applican t wherein at least one of the inputs is chosen via mformation-theoretic feature 
selection and corresponds to a question for which pre-hire applicant responses have been 
collected : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, 

wherein the predictive model is an artificial intelligence-based model constructed from 
pre-hire data electronically collected from a plurality of employees and post-hire data, and the 
model generates its predictive outputs based on the similarity of the inputs to pre-hire data 
collected for the plurality of employees and their respective post-hire data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected from the group consisting of: 

whether the job applicant will be involuntarily terminated, and whether the job applicant 
will be eligible for rehire upon termination. 
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42. (Canceled) 

43. (Previously Presented) The computer-readable medium of claim 37 wherein the 
ineffective questions are identified via an information transfer technique comprising: 

considering various sets of permutations of answers to questions; 
evaluating whether the sets of permutations are effective; and 

determining whether an answer to a question is not in any effective set of permutations. 

44. (Currently Amended) The method of claim i 46 wherein collecting post-hire 
information further comprises: 

tracking whether the applicant has been dropped from payroll. 

45. (Previously Presented) The method of claim 21 wherein: 

the identifying comprises comparing across gender, ethnicity, or age differences. 

46. (New) The method of claim 1 wherein: 

collecting post-hire information comprises receiving payroll information for the 
applicants via a network, determining a termination date for an applicant from the payroll 
information received via the network, and determining tenure of the applicant by comparing 
the termination date with a hiring date of the applicant. 
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